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FOREWARD 



This report was prepared to bring together a number of ideas 
which were presented and discussed at the "Ideas from M-STEP" con- 
ference held at Florida State University on March 27, 1967. A copy 
of the conference program is included as an appendix. "M-STEP" is 
the Iftilti-State Teacher Education Project, a seven-state effort to 
strengthen the contributions of state departments of education to 
teacher education. Consequently, ideas from M-STEP are ideas about 
teacher education at both the preservice and inservice levels. 

The purposes of the meeting, more specifically, were (a) to 
provide an opportunity for reviewing and evaluating ideas which are 
being fostered by M-STEP in terms of their possible contributions to 
teacher education in Florida, and (b) to give persons from the other 
M-STEP states an opportunity to review and evaluate Florida M-STEP 
activities in terms of their own state problems and needs. Persons 
attending included (a) representatives from state departments of edu- 
cation in each of the M-STEP states, (b) representatives from Florida 
teacher education institutions and professional organizations concerned 
with teacher education, and (c) staff members from several divisions of 
the Florida State Department of Education. 

The Iftilti-State Teacher Education Project is attempting to accom- 
plish its objectives through innovative pilot projects being carried 
out in each of the seven participating states (Maryland, Michigan, South 
Carolina, Utah, Washington, West Virginia, and Florida). The M-STEP 
design was adopted to enhance cooperation between the states in planning, 
analyzing, and evaluating the pilot projects and also in disseminating 
information on activities and results. In Florida, pilot projects intended 
to help teacher education institutions and elementary and secondary schools 
to obtain information which they can use ih improving preservice and 
inservice teacher education are now underway. These projects have taken 
two forms: (a) bringing people together in the State to exchange ideas, 

as was done through the conference, and (b) developing techniques for 
analyzing data collected by the State Department of Education to provide 
usable information. Some of these techniques were described at the con- 
ference. 

The Multi-State Teacher Education Project is supported by funds 
granted under Section 505 of Title V of Public Law 89-10 (Elementary and 
Secondary Education Act of 1965). 
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WELCOMING REMARKS 



Floyd T. Christian, State Superintendent 



It is with great pleasure that I welcome this group of dis- 
tinguished educators to our "Ideas from M-STEF” conference. I hope 
that you will find this to be both a stimulating and an informative 
meeting. 

As you know, ideas from M-STEP are ideas about ways to improve 
teacher education. And, as I am sure you realize, the improvement of 
teacher education is of great concern to my staff and to me. 

Now when I say I'm concerned, I don't want you to interpret 
this as meaning I'm worried. I have the utmost confidence in the 
ability of Florida colleges and universities, of Florida professional 
organizations, and of Florida school systems to provide high quality 
preservice and inservice education for Florida teachers. When I say 
I*m concerned about teacher education, I mean that I realize its 
is|>ortance and that I wish for the State Department of Education to 
do its part in helping prepare better teachers. 

Of course, the greatest concern of die State Superintendent is 
the education of the children, youth, and adults of Florida. This State 
has set out to provide every citizen with an opportunity for a quality 
education. And in so doing, we are quick to realize that quality edu- 
cation at aoy level has as its foundation quality teaching. We also 
realize that quality teaching is most likely to be done by persons who 
have been prepared in quality teacher education programs. Therefore, 
in our efforts to improve education in Florida, we are placing a great 
der«l of importance on the improvement of teacher education. We feel 
thteC efforts in this area are likely to pay the greatest dividends. 

The State Department of Education is anxious to help the colleges 
and universities, the elementary and secondary schools, and the organized 
profession to provide better programs of preservice and inservice teacher 
education. We have recently engaged Litton Industries of California to 
conduct a study of inservice education needs in Florida. A report of 
progress to date on this study will be presented at the meeting of the 
Teacher Education Advisory Council next week. We will take into con- 
sideration the findings of this study when making subsequent proposals 
relative to inservice education for teachers. 

We are also working to strengthen the State Department of Edu- 
cation staff in the area of teacher education. The next professional 



position to be added in the State Department of Education will be that 
of teacher education consultant. We will employ for this position an 
individual with broad experience in teacher education. Through his 
leadership and with the help of the Te.< her Education Advisory Council, 
we hope to revise the standards and procedures for state approval of 
teacher education programs. We also hope to establish more effective 
communication and collaboration between teacher preparation institutions, 
elementary and secondary schools, professional organizations, and the 
State Department of Education. 

Frcxn these remarks, you can see that State Department of Edu- 
cation efforts to improve teacher education in Florida are moving in 
two directions. First, we are trying to obtain infonnation which will 
enable us to make decisions on a more rational basis. In colleges and 
universities this is done through institutional research. The State 
Departirent of Education is also carrying out institutional research. 

This is exemplified by the study of inservice education. The second way 
we are working to improve teacher education is by promoting greater com- 
munication, cooperation, and collaboration between colleges and univer- 
sities, professional organizations, elementary and secondary schools, and 
the State Department of Education. 

Participation in M**STEP has enabled us to move more quickly in 
both of these directions than we might otherwise have done. Through 
M-STEP we have developed an information retrieval system which enables 
us to obtain from our current files a great deal of information which 
was not previously accessible. Fred Daniel and Wil Robinson tell me 
that if there is information in those files which I need, they can have 
it for me in the morning. Through M-STEP we have also conducted extensive 
analyses of teacher evaluation data. These will be discussed this after- 
noon. 



M-STEP has also served to stimulate greater collaboration between 
the Florida State Department of Education and other agencies concerned 
with teacher education. Representatives of the Association for Student 
Teaching worked with Teacher Education Advisory Council members and State 
Department of Education staff members in the development of the booklet. 
Guidelines for Student Teaching in Florida, an M-STEP sponsored publication. 
Assistance to colleges and to graduate students who may wish to make us of 
State Department of Education data files has been made available through 
M-STEP. On Tuesday, Wednesday, Thursday and Friday of this week, regional 
conferences dealing with school-college cooperation in professional labora- 
tory experience will be held in four Florida cities. These are being 
sponsored by the State Department of Education and the Association for 
Student Teaching. 

However, as Dr. Bosley will probably point out, M-STEP is not intended 
sinq>ly to promote activities within states. It is also designed to stimulate 
the interchange of ideas between states. During this past year, members 
of my staff have had the opportuni*:y to visit several of the M-STEP states 
and to discuss with them their teacher education activities. We are pleased 
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.today to have the opportunity to exchange ideas with representatives of 
each of the other states in the Project. I hope that finding out 
first-hand what is being done in these states will provide ua with new 
insists which will result eventually in improved teacher education in 
Florida . 

Let me conclude by reiterating my welcome: to college repre- 

sentatives, to representatives of professional organizations, to State 
Department of Education staff members, and especially to our out-of-state 
guests. We are pleased to have you here. 



L ERIC 
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PART ONE 



m-step in other states 



1 



o 

ERIC 



PRESENTATIONS BY M-STEP REPRESENTATIVES 
FROM OIHER STATES 



The persons participating in this conference were asked to pro- 
vide a statement outlining the M-STEP idea or ideas which they would 
describe. This statement was to point out the type of problem toward 
which particular M-STEP activities are directed, the general approach 
which is being taken, and the general rationale for adopting such an 
approach. Those statements are repT'jduced below. 



" The M-STEP Idea " 

M-STEP is a compact of seven states, formed for the purpose of 
finding ways to improve teacher education. 

M-STEP 's prime goal concentrates on the improvement of laboratory 
experiences in teacher education. Though M-STEP advocates no particular 
design, this concept could involve several stages and types of clinical 
processes, among them being: 

1, Studen*: teaching, standard model 

2. Student teaching, deluxe version 

3, The clinical experience concept: a broader approach to a genuine 

laboratory situation. 

a. undergraduate internships 

b. graduate and inservice internships 

4. An intensif. ad clinical core program in which professional labora- 
tory experiences cease to function as an adjunct to teacher pre- 
paration, and become a major function. This level is characterized 
by: 

a. A clear-cut and thoroughly defined set of teaching skills, 
techniques, understandings and items of professionally oriented 
awareness which comprise the components of effective professional 
performance. 

b. These components of effective professional performance should 
be organized into a system of priorities and sequence, and 
their opportunities for acquisition extended over a larger 
period of time than is now devoted tc laboratory experiences. 
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c. The professional learning processes of a direct nature are 
appropriately balanced with reorganized course work, seminars 
and discussion groups. Both avenues of professional develop- 
ment are supplemental and reinforced with latest visual media 
and processes. 

d. The student's progress through the professional sequences of 
the four-year college program would be dependent upon, and 
guaged by his success in the acquisition of the teaching com- 
ponent series rather than by his accumulation of semester 
hour credits. 

5. A clinical experience center, operating as an autonomous entity, 

but guided by some type of control system. Presumably such a unit 
would serve a public school system of a state or region, and would 
also provide professional laboratory services to one or more colleges 
and universities. 

A second major tract in the M-STEP is the exploration of effective 
means for utilizing video tapes and video processes in teacher education. 

The third major track involves the organization of intra-state 
cooperative effort between professional agencies and organizations within 
the state. 

A fourth item of M-STEP concern is the creation of operational proto- 
types for interstate cooperation in teacher education. This is one of the 
original thoughts of M-STEP, as were the other three, because somehow in 
American education state boundaries have tended to impede the transfer of 
ideas. 



Howard E. Bosley, Director, M-STEP 



" Videotapes for Teacher Education" 

M-STEP planning and programming is proceeding along two major avenues 
both of which are related to the original objective of strengthening and 
improving the preservice education of teachers especially in the area of 
student teaching or laboratory experiences. 

One major effort is being directed toward the publication of a 
Handbook on Student Teaching. This is considered to be a highly desirable 
project and one that complements the other major effort in M-STEP at this 
time. Regular meetings are being held by college and State Department 
personnel to compile data and resolve any problems which might arise in the 
process of developing an effective publication. The proposed handbook is 
undergoing continuous revision and a final draft is expected to be adopted 
for printing and preliminary testing and use beginning July 1 of this 
The publication is expected to result in a strengthening of the leadership 
role of the State Department with the twenty-two colleges in the State and 
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la bringing about improvement in planning for and carrying out better 
student teaching programs in the various colleges concerned. 

The second major effort is being directed toward the production of 
several video tapes which deal with various phases of the student teaching 
process. These tapes, together with the handbook would constitute a 
major breakthrough in making available to those involved in student 
teaching both printed and visual materials. The aim of such materials 
is to help such persons and institutions transmit not only the fundamental 
theories and principles of good student teaching programs but to provide 
audio-visual tapes which effectively demonstrate that such principles 
actually work. It is hoped that both theory and practice are being wedded 
in these tapes. 

Completed or in process at this time are tapes on lesson planning 
(1), unit planning (1), debate (1), the use of media teaching (1), methods 
and motivation in the classroom (3), laboratory teaching (1), and the 
roles of the various persons involved in student teaching (1). All of 
the above have as their goal the employment of methods which can be used 
with futute persons as teaching materials to improve the quality of student 
teaching programs. 



Boyd Israel, South Carolina 



Utah is focusing on the use of instructional media to Improve labora- 
tory experiences for pre-service and in-service teachers. An attempt is 
being made to improve teacher education through more extensive use of video- 
taped observations (recorded classroom episodes) and micro- teaching. 
Following a planning conference held at Park City last summer, guidelines 
were developed and pilot projects outlined to emphasize new uses for 
instructional media. A series of 13 recorded classroom episodes were pro- 
duced, and these will be distributed and evaluated in an effort to determine 
the extent to which they can supplement, extend, and reinforce laboratory 
experiences . 

Concurrently, two project activities have been undertaken which 
involve experimentation with micro-teaching as a technique in teacher edu- 
cation and one activity concerned with "The Dynamics of Team Teaching." 

The micro- teaching project being conducted in cooperation with the Uni- 
versity of Utah is directed toward the identification of certain teaching 
behaviors utilizing videotaped peer-teaching sequences and the production, 
evaluation, and coding of the peer-teaching episodes. In conjunction with 
Brigham Young University, an attempt is being made to (1) identify criteria 
for model videotapes foL use in micro-teaching which will demonstrate 
strategies for effective teaching behavior; (2) screen presently available 
videotapes and films on micro- teaching for inclusion in a bank of model 
tapes and films; (3) prepare model tapes for demonstration of micro- 
teaching techniques when they are not available; and (4) prepare video- 
tapes of model teaching behavior to be used in micro-teaching sessions. 

The team teaching activity involves production of a series of two or three 
videotapes in cooperation with Weber State College and the Weber County 
Exemplary Team Teaching Center. 

Vere A. McHenry, Utah 
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“ Centers for Student Teaching ** 



In Michigan there appears to be an established need for agree** 
ment among che teacher education institutions and the sponsoring school 
districts as to the nature and extenc of the student teaching experience. 

Conditions in different geographical areas of the State vary 
widely with respect to the nature and number of population, resources, 
number of teacher education institutions and other factors. For this 
reason, we have decided to divide the state into a proposed half dozen 
regions for better administration of student teaching programs. In each 
region, problem areas will be identified and discussed, committees will 
be appointed to study such areas as, selection and training of the local 
school and college supervising teacher, contractual agreements among the 
institutions and the local schools, policies on placement of student 
teachers, financial assistance to the local school supervising teacher 
and other problems that might be identified through the regional approach. 

Michigan feels that through the democratic cooperation of colleges, 
universities, local schools, professional organizations, and the State 
Dep.rtment of Education that student teaching programs can be improved. 
Emphasis on experimental programs in student teaching will be stressed in 
each of the six proposed regions. 

In order to coordinate the activities of the various regions and 
isiplement the overall project in Michigan, a Reaction Panel composed of 
representatives from the colleges and local schools in the six regional 
areas has been appointed. This group will meet regularly with Michigan 
M-STKP personnel in carrying out objectives of the project and also will 
be responsible for suggesting new ideas and designs for professicmal labora** 
tory experiences in student teaching. 

As part of the M-STEP activities in 1967, Michigan will host a three 
day Clinic on Student Teaching, May 31, June 1 & 2, 1967. Each M-STEP 
Director and Coordinator will be invited to Michigan to participate in a 
conference with representatives from each of the 25 teacher education 
institutions in the State. This clinic will provide the opportunity to 
exchange ideas and react mutually to the various state programs in student 
teaching and teacher education. 



Jerry E. Chapman, Michigan 

Underlying the West Virginia M-STEP program is the belief that the 
cooperative efforts of public schools, the State Department of Education, 
teacher education institutions with diverse student populations, programs 
and purposes, will produce Improved student teaching experiences. 

Based on this thinking, M'STEP in West Virginia has taken the form 
of a Pilot Center for Student Teaching located in the public schools of 
Kanawha County. The cooperating institutions include Marshall University, 
a state university at Huntington; West Virginia State College, formerly 
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non-white, at Institute; West Virginia Institute of Technology, a 
technical school with a secondary teacher preparation program, at 
Montgomery; Concord College, a multi-purpose college at Athens; 
and Morris* Harvey, an independent college located in Charleston* 

The Center, which in its initial session enrolls approximately 
thirty (30) students from the five colleges, is designed to operate 
under a staff of professionally trained personnel holding positions 
on the public school staff. Direction is being given to this staff 
by an Advisory Committee composed of representatives from the cooperating 
teacher training institutions, the public schools and the State Depart- 
ment of Education. 

The cooperating agencies channel their resources through the 
Center to the participants through seminars for student teachers and 
an intensive inservice program for supervising teachers. 

It is believed that both a climate for the improvement of student 
teaching, and the organizational pattern for implementing such improve- 
ment, will result from the Center approach being carried out in West 
Virginia. 



John B. Himelrick, West Virginia 



Maryland M-STEP is developing a center for laboratory experiences 
in teacher education. Project activities are focused in Kemp Mill 
Elementary School, Silver Spring, Maryland. Policies for the operation 
of the center are determined by an M-STEP Steering Committee, and 
cooperatively administered by representatives of the Montgomery County 
Public Schools, the University of Maryland, and the Maryland State Depart- 
ment of Education. 

The general objectives for the Project are: 

1. To demonstrate that preservice education and inservice staff 
development can be unified in a continuing teacher education pro- 
gram when there is increased responsibility for preservice edu- 
cation by the school system and for inservice education by the 
college in a teacher education center. 

2. To identify and study new roles and skills for public school and 
college faculties in a continuing teacher education program. 

3. To identify and study the role of a teacher education center 
coordinator jointly appointed and employed by the school system 
and college. 

4. To identify and study the role of a state department of education in 
a continuing teacher education program. 

Experiences for student teachers will be planned cooperatively by 
representatives of the Maryland State Department of Education, the Montgomery 
County Public Schools anu the University of Maryland. The University Center 
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Coordinator will work directly with the students and cooperating teachers 
to utilize the strengths of all faculty neiid>ers so that a wider variety 
of experiences than has previously been possible can be provided. 

Arthur P. Kulick, Maryland 



^ 'Coordinating Preaervice and Inservice 
Teacher Education Prograns" 

i* ’ The concept of teacher education needs to be extended. The pre- 
paration of the teacher begins early in his collegiate career and 
extends into the first several years of teaching. The education 
of the teacher really never ends. 

2. The responsibility for teacher preparation is shared. Primary 
responsibility for preparation gradually shifts from the teacher 
preparing college to the public school system, to the professional 

: ^ - associatioh, to the individual himself. 

3. The eonoept of what teaching is about is changing. The differences 
between public speaking and teaching are becoming clearer. Mew 
a^ia provide the possibility for individualization of instruction. 

• Ifethods^eiiphasizing pupil initiative and pupil responsibility for 
learning seem to be most promising. 

4. - ' ■ ^ ^ practical phases of teacher preparation will be done 

in -phbbie adiooU involving district personnel and college people. 

5. Teacher preparation would be -greatly improved if preaervice and 

1 '^"imaervioe proAkasm were articulated, and if the resources of both 
:'tlia odllage and the district were used. 

6. Teacher preparation should be designed for excellence. 

William H. Drunmond, Washington 
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"CAN THE M*STEP IDEAS BE ADAPTED TO FLORIDA'S NEEDS?" 



Following the presentations by M-STEP representatives from out 
of state, conference participants were divided into discussion groups 
to consider the question, "Can the M-STEP ideas be adapted to Florida's 
needs?" As is often the case, more questions were asked than were 
answered. The statements below were taken from recorders' reports (and 
edited). These statements are included to provide an indication of 
group interests and reactions. (The names of the members of any particular 
group can be obtained from the complete list of participants included in 
the Appendix. The number in parentheses after each name indicates the 
group in which he participated.) 



Group One 



Chairman: Roy E. Dwyer Recorder: John Waters 

This group posed two questions: 

* 1. Should the preservice instruction be "cleared up" before 

student teaching and inservice instruction are tackled on a broad scale? 

* 2. Should there be a state coordinating council to consider the 

problems in student teaching programs? Of inservice education? 

Group TWO 

Chairman: William Maloy Recorder: M. Mitchell Ferguson 

Group two discussed the Importance of developing cooperation 
between local schools, teacher education institutions, and the State Depart- 
ment of Education in improving preservice and inservice teacher education. 

One agency cannot solve all problems alone. 

The discussion also dealt with the importance of classifying types 
of teaching performance and developing techniques for appraising them. A 
third area of discussion was the potential use of video tapes in local schools. 



Group Three 

Chairman: Evelyn Sharpe Recorder: William E. George 

Group three felt Florida could profit from accepting parts of the 
M-STEP ideas or suggestions made by the other states. They did not want 
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the total idea accepted and implemented. A large portion of the dia- 
ctiaaion centered around the internship program and how these ideas could 
be used to assist Florida in receiving better trained interns in greater 
numbers. The role and responsibility of the university, State Depart- 
ment of Education, county staff, and professional organizations for intern- 
ship program and in-service education was also discussed. The 5-year teacher 
education program was discussed, along with ideas on certification in 
general. Simplicity and flexibility in teacher certification, as dis- 
cussed .by Dr. Drummond, was explored relative to Florida needs. The 
principle, in theory, sounded good; however, the group felt a much closer 
exaoiination was needed before the idea was implemented. 



Group Four 



Chairman: Bert Sharp Recorder: John Ritter 

The discussions of this group centered around four questions. The 
questions were, 

1. Are any M-STEP projects aimed at identifying good instructional 
practices? If so, what criteria are being used? 

2. What do the people in the Washington project mean by broad 
field of training? Do the concepts of hours vs. credits appear anywhere 
in the plan? What kinds of specializations are included in the training 
of elementary teachers? 

3. In more specific detail what are the arrangements between 
local systems and the University in the Maryland plan for a training center? 

4. What kind of cooperative arrangements are being made by the 
ins t i t ut ions in Michigan? 



Group Five 



Chairman: Sister Ann Thomas Recorder: W. W. Wharton 

In the group discussion it was the consensus that in the total 
program of teacher training the local school districts must assume a 
greater responsibility. How local d.istricts can prepare for this responsi- 
bility and where the leadership will come from appeared to be major problems 
to be resolved, along with how to finance the local district effort. 

The group felt that responsibility for the supervision of interns 
should remain with the colleges. However, the role of the colleges and the 
local school districts should be re-examined and perhaps redefined. The 
accomplishment of the goals of teacher education, may require the addi- 
tional leadership and direction of a third agency. It was suggested that 
the State Department of Education was a logical and desirable source of 
leadership along with the colleges and local districts in planning 
and directing a program of teacher training. The point of a cooperative 



effort made by the colleges, local school districts and State Depart- 
ment of Education was emphasized. 

It was pointed out that because Florida must recruit a large 
nund>er of teachers from outside the State, the development of any pro- 
gram must provide for inservice training as well as preservice training. 




PART TWO 



THE FLORIDA M-STEP PROGRAM 



I 
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USING INFORMATION TO IMPROVE 
STATE EDUCATIONAL LEADERSHIP 

Vhn. Cecil Golden, Assistant Superintendent 
Teacher Educatipn, Certification and Accreditation 



We asked for the privilege of being a part of M-STEP because 
of its major purpose: to strengthen the capacity of state departments 

of education for providing leadership in the development of joint 
responsibility among local education agencies and teacher education 
institutions in the preparation of professional personnel. 

The traditionally accepted roll of state departments of 
education in teacher education can be divided three areas: (1) to 

approve institutional programs of teacher education, (2) to issue 
certificates to instructional and administrative personnel, and (3) to 
accredit schools. 

Historically^ state departments of education have devoted 
more time to the administering of these functions than they have to 
appraising their effectiveness and working toward the in^rovement of 
teacher education. M-STEP is attempting to place renewed emphasis on 
the latter. A major purpose of the Project is to promote "joint 
responsibility among local education agencies and teacher education 
institutions for the preparation of professional personnel." And it 
is with the help of state departments of education that this is to 
come about. 

Is this a desirable function of a state department of edu- 
cation? Is this a logical function of a state department of education? 

We think the answer to both questions is, yes. 

The next question — Why haven't most state departments of edu- 
cation directed more of their energies and resources toward achieving 
this purpose. There are of course many reasons. In Florida we have 
decided to explore one factor which in our opinion offers good possi- 
bilities for the State Department of Education to exercise a greater 
leadership function in teacher education. 

"n the process of administering the education affairs of a 
state, the State Department of Education either collects or can collect 
data of almost unlimited amounts on many phases of education. One of 
the primary objectives of our project is to develop techniques or systems 
which could be employed by state uepartments of education which will ensure 
the collection of appropriate types of data in usable form about teaching 
and teachers in the state. Then through effectively organized procedures, 
we hope to show ways of feeding back this information to those who administer 
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and carry oqt educational services within the State. The following 
are some tasks we are now working on with these goals in mind: 

1. We are analyzing procedures for appraising teacher performance. 

2. We are analyzing results of teacher examinations to determine 
strengths and weaknesses in the preparation of teachers inservice. 

3. We are analyzing teacher assignment practices. 

4. We are promoting school-college cooperation in providing labora- 
tory experience in teacher education. 

5. We are studying the migration patterns of teachers. We will com- 
bine with this anticipated curriculum change and develop a detailed 
projection of teacher needs for the next 10 years. 

With appropriate data on all phases of education being continuously 
fed back to the proper organization and agencies, we anticipate continuous 
modification and change in service provided. 

The rationale underlying our present approach goes something like 
this: Educational leadership is a process intended to promote changes 

which will facilitate the attainment of educational objectives. It is 
feedback that calls attention to changes which need to be made. If those 
who exercise control over our educational services receive no information 
relative to the effectiveness of their operation, it is probable that no 
change will be initiated. (No leadership will take place). 

We believe that for a state department of education to exercise 
leadership it must develop "organizational inertia." The State Department 
of Education must see that there is "mutual interaction" among the people, 
organizations, and agencies which are to provide services which will improve 
teaching. A basic role of the State Department of Education should be to 
identify types of changes needed and to facilitate feed back to evaluate the 
changes when they are put into practice. 

We are very encouraged about the possibilities of M-STEP and we 
are confident that the role of the State Department of Education as it 
relates to teacher education is definitely moving in the direction of the 
major goal of M-STEP. 
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THE FLORIDA STATE DEPARTMENT OF EDUCATION INFORMATION SYSTEM 



by L. Everett Yarbrough 
Director, Systems 

In the State Department of Education we believe that accuracy 
and relevance are the qualities which make information significant and 
useful. Regardless of their internal speeds or technical efficiency, 
computers and other types of data processing equipment cannot be 
effectively utilized by an organization unless such equipment is appro- 
priately integrated with other elements of the information system of 
the organization. Hence, we are now developing a comprehensive infor- 
mation system designed to utilize information from a wide range of 
sources and to generate information which is relevant to a wide range 
of problems. 

Department organization for systems development and imple- 
mentation is shown in this Figure 1. The solid lines on this chart 
represent the administrative channels of the state department in relation- 
ship to the information system. The chief administrator of the infor- 
mation system is the deputy superintendent who also has general adminis- 
trative responsibility for. other department -wide services. Both the Data 
]Processing Operations Section and the Systems section are responsible 
directly to the deputy superintendent. Administration of the information 
system at this level has been a significant contributing factor to the 
ability of the department to make decisions necessary to reasonable pro- 
gress in moving from an uncoordinated information system toward the ideal 
of a coordinated system. 

Although administrative responsibility for the information system 
flows through the channels represented by unbroken. lines on this chart, 
there are many relationships of the information system personnel repre- 
sented by broken lines oh the chart. These, relationships are best described 
as coordinative - consultative. Note that relationships are indicated 
between information system personnel and those of each division of the 
department and local school personnel. Note also the relationship between 
the Department Data Systems Steering Committee and the organization for 
administration shown here. 

A coordinated information system (Figure 2) is characterized by 
interrelated procedures making optimum use of electronic and mechanical 
equipment. Input documents are designed to (1) minimize duplication of items, 

(2) provide data as a by-product of an on-going activity at the source, 

(3) utilize standard item definitions drawn from catalogues of items developed 
from theoretical consideration of needs and evaluation of needs that become 
apparent in practical implementation. The data are processed through 
interrelated procedures which are in fact one overall set of procedures 
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allowing for output documents to contain infonnation entering the system 
through any of the input sources. A continuously updated cumulative 
record of all items of data is maintained. These items are organized 
around logical poles such as a pupil, a teacher, a facility, etc., with 
the identification number of the pole serving as the major control for 
the record. 



Integration of the system segments primarily concerned with 
the functions of payroll reporting, state aid distribution, teacher 
certification and school accreditation is Indicated in Figure 3. It 
may also be seen that other segments are related to these segments at 
several points. In fact, this figure represents only four of more than 
50 major segments of the educational information system segments under 
the immediate direction of the State Department of Education. Since the 
Florida Educational Information system is a statewide system, however, 
there are many segments operated by county units or universities which 
interact on a coordinated basis with those directed at the state level. 

Continued development and implementation of practical appli- 
cations is depicted in Figure 4. The directions which this would take 
would be determined by obvious needs as well as theoretical considerations. 
Data committees may be expected to move the information system of an 
organization from a typical traditional uncoordinated status to an adequate 
coordinated status in steps similar to the following: 

1. Initial desired output document contents are identified. 

2. A determination is made of the contents of the data bank needed to 
produce the desired output. 

3. Data gathering instruments are developed to provide source data 
required . 

4. The contents of a second desired output document are defined. 

5. A determination is made of the data required to be in the data bank 
to produce the desired output. 

6. A determination is made as to those needed items already in the data 
bank. 

7. Data gathering instruments are developed to provide additional 
items required. 

8. Additional output requirements are defined, required data bank 
contents determined, new data to be gathered are provided until 
that happy day when for each new output requirement defined 
adequate data are available in the data bank and additional 
data gathering instruments and procedures are not required. 
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INFORMATION ON TEACHERS AND ON SCHOOLS NOW COLLECTED 
BY THE DIVISION OF TEACHER EDUCATION, 
CERTIFICATION AND ACCREDITATION 



From the beginning of M-STEP, the large body of data which 
have been routinely collected by the Florida State Department of Edu- 
cation has been considered a significant resource. This information 
is particularly valuable because much of it is stored on magnetic tape 
and is available for selective retrieval via computer. Florida M'STEP 
has placed particular euqphasis upon developing methods for using infor- 
mation collected by the Division of Teacher Education, Certification 
and Accreditation. Hie type of information in the tape files is described 
below. 



In files which contain teacher records, the teachers are 
identified by a State Department of Education identification nuni>er assigned 
either by the Certification Section or the Teacher Retirement Service. In 
files which contain school records, the schools are identified by a county 
identification number and also a school number which is assigned by the 
School Plant Section. 



Master Certificate File 

Information on the master certificate file is collected by the 
Certification Section. This file contains a record for each teaching 
certificate which has been issued. Each record contains certain bio- 
graphic or status information on the certificate holder, as well as infor- 
mation about the certificate. The following is information on certificate 
holders contained on the file which may be relevant in certain types of 
studies: 

Name 

Birthdate 
State of birth 
Sex 

Marital status 
Institution of graduation 
Year of graduation 
Degree 

The following information about the certificate is also included: 

Certificate type (There are twenty-two different types of 
certificates which are issued or have been issued.) 

Academic rank (The rank represents the level of the academic 
degree on which the certificate is based.) 

Year of issue 
Expiration date 



Minimum Foundation Program salary rank (This determines the 
level of state support under the Minimum Foundation 
Program to which the county employing this certificate 
holder is entitled for his services.) 

Endorsements (These indicate the specific subjects or fields 

to which this certificate holder is eligible to be assigned.) 



Teacher Examination Scores 

Candidates requesting teaching certificates since 1961, have 
been required to file with the State Department of Education a score on 
either the National Teachers Examination or the Graduate Record Examination. 
The complete file of these scores is on magnetic tape and available for 
analysis. 



In addition. Educational Testing Service provides the Department 
with records of scores earned by all persons taking the National Teachers 
Examination in Florida centers. These scores are also available in a 
separate file. However, the individuals who took the tests are not 
identified on the file. The only identification information refers to the 
individual test centers. 



Teacher Evaluation Records 

Since 1961, it has been required by statute that each teacher be 
evaluated annually and that this evaluation be placed on file in the State 
Department of Education. The evaluation records from 1963-1966 are stored 
on tape. However, these records are available only to State Department of 
Education personnel and authorized county personnel. It should be noted 
also, that interpretations which can be made from these evaluations are 
limited. This matter is discussed in a subsequent section of this report. 



School Information 
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In administering the school accreditation program, the Accredi- 
tation Section collects considerable information on public schools (grades 
K-12). The following information has been compiled from accreditation reports 
for the school years 1961-67 and is stored on tape. The file contains one 
record for each school. 

School name 
Grade organization 
Type of school 
New school 

Operation less than 3 years 
Double sessions 

Number of instructional staff members 
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Number of library clerks 
Number of guidance clerks 

Number of secretaries (other than library and guidance) 

Number of teacher aides 

Number of custodians and maids 

Number of library books 

Number of library books per child 

Membership, K**6 

Net length shortest day, K'6 

Net length longest day, K-6 

Method of Instruction, elementary 

Number of elementary departmentalized periods 

Membership, 7-9 

Membership, 10-12 

Total membership, K-12 

Met length of shortest day, 7-12 

Maximum net minutes per period, grades 7-9 

Maximum net minutes per period, grades 10-12 

Maximum net minutes per period, grades 7-12 

Number of instructional periods per day, secondary 

Number of acres In site 



Teacher Assignment Information 

Information on teaching assignments for the school years 
1964-1967, has also been collected by the Accreditation Section. The 
assignment file contains a record for each teacher In each school. Itie 
record contains the following Information: 

Special permits which have been granted (for a teacher with 
an assignment not covered by his certificate but who has 
earned at least six semester hours of credit toward Infield 
certification. ) 

Full-time or Part-time teacher 

Assignment of teacher by course and period (This Indicates 
the specific subject (s) and level(s) taught by each 
teacher or the type and level of assignment held by 
an administrator.) 

Infield certification (l.e., Is the teacher or administrator 
properly certified for his assignment) 

Memberships (l.e., number of pupils) by period assignment 

Number of planning periods assigned to the teacher 

Number of supervisory periods assigned to the teacher 

Predominant type of Instruction for each assignment (types 
Include regular Instruction, core or block time for 
basic education classes, language laboratory or science 
laboratory, large lecture, programmed learning, team 
teaching, large class television, and service personnel.) 
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Aiuilviis Which Have Been Conducted by the Accreditation Section 

In the Accreditation Section, the following types of Infor- 
mation are available for the years 1963-64, 1964-65, and 1965-66: 

Item analysis of standards for school, county, and state, 
showing percentage of compliance with standards. 

Total courses, classes, students, and teachers by school, 
county, and state- 

a. Total classes taught (by courses) and size of 
classes 

b. Total students enrolled In classes (by courses) 

c. Total teachers, by course assignment and (In-fleld and 
out-of-fleld) certification 

Total of high school graduates# by school, county, and state, 
entering colleges, universities, technical, trade, and 
other schools In and out of the state. 

Analysis of selected Items comparing statistical Information 
to personnel Information. 

Staff assignment analysis of "out-of-fleld** personnel, showing 
teaching certification and teaching assignment by period. 
Number of each type Of audio-visual equipment, tapes, records, 
and films housef*. In the school. 



USING THE INFORMATION WE HAVE 



K. Fred Daniel and Wll C. Robinson 



A major goal of Florida M-STEP is to make immediate use of 
information now on file on the State Department of Education- -parti- 
cularly that collected by the Division of Teacher Education, Certi- 
fication and Accreditation. It. is felt that within this bank of 
data is information which is relevant to numerous current programs 
and problems. 

For example, teacher examination scores might be analyzed to 
determine areas of strength and weakness in the academic preparation 
of various segments of the teaching population. Teacher assignment files 
mi^t be analyzed to determine the teaching locations and assignments for 
graduates of various institutions. The institutions might then use this 
information to follow up their graduates to obtain additional infor- 
mation for evaluating their programs. Teacher examination files could 
also be analyzed to study teacher employment and migration patterns or 
to obtain information on which to base projections of future teacher 
needs . 



In addition, the files could be used to select samples for 
researchers interested only in a given segment of the teaching population. 
Examples might be teachers of a given subject, teachers who graduated from 
a certain institution, teachers with certain types of degrees or certi- 
ficates, or teachers teaching in a certain type of school. 

Almost all of the information collected in the Division of Teacher 
Education, Certification and Accreditation which is of potential value 
for purposes such as those described above is presently in a form which 
makes it accessible by computer. There are also data collected by other 
Divisions which could be combined with this information to make it even 
more useful. 

There has been a problem, however, in using such information. When 
the many data files in the Department were built, retrieval programs were 
not developed which would make the information immediately available for 
purposes such as those described above. It was not--nor is it now--feasible 
to write individual retrieval programs for the multitude of potential 
problems to which certain portions of these data may be relevant. Thus, it 
has been necessary to design a new approach. 



A Flexible System for Data Retrieval and Analysis 

The approach adopted consists of a system (i.e., a series of 
integrated programs) designed to select, combine, edit, analyze, and print 
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an almost infinite number of different combinations of information 
froiB almost any collection of data files * With this system it normally 
would take an hour or less to prepare all of the necessary Instructions 
to run the data retrie/al and analyses for a given problem using the 
State Departpent of Education coiq>uter8. In certain cases the instructions 
coul4 be prepared in a few minutes; In others the time required might be 
several hours. In all eases, however, the necessary preparation time is 
a minute fraction of that which would be required to write a special 
program for the job. 

the flexible system for data retrieval and analysis is depicted 
in Figure 5. The system is still being developed. However, there are 
now in use ih the Department programs which perform each of the functions 
inclhded ih tt^ system. Thus, it is presently possible to perform most 
of the rattihiral ahd analysis operations which might be desired. In using 
the presatil ptogriMSa with aattain iobs it is necessary to perform time con- 
suming additional opayations in order to obtain the desired results. Hence, 
while the sfateai i» its present from makes it possible to obtain much infor- 
matioo whten wda hot immediately available in the past, for some Jobs it 
is extremely inefficient. This limits the application on which the system 
can be used. 

The flexible retrieval and analysis system makes use of two com- 
puters: an IBM 1401 and an RCA 301. While it is not necessary to have 

two computers to perform the functions of this system, the availability of 
programs which make up the system and the availability of data processing 
facilities in the State Department of Education made this desirable in the 
present situation. Most of the data files compiled by the Division of Teacher 
Education^ Certification and Accreditation are built using the 1401 computer. 
Therefore, the programs for selecting and combining information are written 
for the 1401. However, most general analysis programs available in the Depart- 
ment are written for the 301. Therefore, the analysis phases of the system 
use the 301. 

The following is a brief description of each of the programs or 
functions in the general system with information on its present stage of 
development . 

Select / Search Program 

This program will select from any file, designated items from all records 
on that file which meet .certain designated criteria. The criteria for 
selecting records and the items for selection are specified for each Individual 
run. The criteria on which the selection is to be based must of course be 
included on the file, (i.e.. If all graduate of a given institution holding 
rank II certificates are to be selected, both the institution of graduation 
and the rank of certificate must be coded somewhere on each record.) Criteria 
may be stated in the form of ranges or discrete values. (An example of a 
range criterion would be *'all teachers with birth dates from 1910 to 1923”; 
examples of discrete criteria would be "teachers in counties 10, 27, and 54" 
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Program to Edit Tape for RCA 301 
Foreran System 






I Tape inputs to the fortran system (described below) must be written 
in ^ special format. This program performs the necessary editing to make 
tap#s not in that format acceptable to the fortran system. Is is now fully 
operabli 



Geniralj Print Progfaiii 

I ^ f ■ 

I program will prepare a printed list containing selected information 

froiji each fSboTd'^Ohli file. The specific information to be selected and the 

for^ in whjLch it is to b|e printed is specified for each individual run. * Appro- 
priate hea|lings are printed at the beginning of the report and at the beginning 
of tach page. No, progralns are presently being used in the State Department 
of Aducatiw which. will perform this function as described above. There are 
programs, however, which will print all information in a given file (i.e., a 
*'tabe dump*') or which will print information in certain specified formats (but 
only . if the. input is in a certain specified format). 
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program to Convert T.-»pe from IBM 1401 
Coding ^ RCA ^ Coding 

The programs for selecting and combining information utilize 
the IBM 1401 computer. However, the Department programs for analyzing 
data are written for the RCA 301 computer. Thus, it is necessary to 
convert tapes to RCA 301 coding before analysis. The conversion pro- 
gram performs this function. It is now fully operable. (If the input 
to the RCA 301 programs is in the form of cards, no conversion is necessary.) 



RCA 301 Fortran System 

Fortran is a programming language which is particularly con- 
venient to use when the operations to be performed on the data are mathe- 
matical in nature. This system is now available and fully operable in the 
State Department of Education. 



General Tabulating Program . 

This program analyzes data and places the results in the form 
of tables of two or three dimensions. It will accept data in virtually 
any format from cards or tape. It is now fully operable. 



Program to Edit Data for General 
Statistical Analyses 

The statistical analysis programs described below require that the 
input data be in rigidly specified formats. This program is designed to 
perform the necessary editing. At present, this program is in the planning 
stage. The editing for the statistical analysis programs is now done by 
hand coding, re-formating with offline equipment, or with fottran programs. 
All of these methods require that input to statistical analysis programs 
be in the form of punched cards. 



Statistica l Analysis Programs 

These programs perform a number of standard statistical analyses such as 
means, standard deviations, simple correlations, multiple correlations, 
regression analyses, equation solutions, analysis of variance, and analysis 
of convar lance. They accept inputs in the form of punched cards, magnetic 
tape, or paper tape. These programs are fully operable. However, they can 
be used only when the input data are especially edited. (A program, described 
above, is now being prepared to provide this editing in an efficient manner.) 
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Assiatancf. In Using the System 



A major goal of Florida M**STEP Is to make State Department of 
Education data available to persona who can put this Infomiatlon to use 
In advancing education. This Includes people both In and out of the 
State Department of Education. The assistance available consists of 
consultation regarding retrieval and appropriate usage of Information 
from Department files. Project personnel are available to discuss 
problems and to prepare the necessary instructions for retrieval and 
analysis problems. At the present time, no charge Is made for 
consultation or computer time. 



ANALYSES OF THE FLORIDA TEACHER EVAUIATIOM DATA 



K. Fred Daniel* Director 
Florida M-STEP 



Florida Statutea^ require that each certified achool employee 
(teacher, adainiatrator , etc.) be evaluated annually and that thia 
evaluation be placed on file in the State Departaent of Education. A 
atandard fora to be uaed for the evaluationa haa been adopted by the 
State Board of Education. Thia fora ia coapleted in triplicate with 
one copy for the State Departaent of Education, one for the county 
office, and one for the filea of the local achool. The evaluationa 
are notaally coapleted by principala and counteraigned by county 
auperintendenti. In the State D^af»rat of Education, they becoae 
part of teacher peraonhel filea. They are available for Inapection 
only by State Departaent of Education peraonnel and by county achool 
personnel on official business. 

The evaluation fora is designed so that it can be reed with, 
and optical scanner. The ratinjp are thus transferred autoaatically to 
punched cards. The punched card data are transferred to aagnetic tape 
for retrieval and analydis via coaputer. This haa aade possible the 
auaaarising of ratings for individual schools, for counties, and for 
the entire state. Statewide sunaaries of ratin s for the years 1963-64 
and 1964-65 are presented in Tables 1 and 2. 



Analyses Using Earlier Versions 
• of the Fora 

Data froa the Florida Tbacher Evaluaticai lOra have been 
used by a madier of Florida State Uaiversity students in their graduate 
thesis and dissertation projects. Studies of this type which have been 
coapleted to date aade use of an earlier version of the evaluation fora. 
This version differs froa the present in that it contains aore itcas and 
eaploys a scale with five response positions, rather than three. The 
nature of the iteas on the new and old foras is the saae, however. 

liurzbach^ studied the relationship between principals' ratings 
and three status factors: degrees held, teaching experience and county 

of eaplpyaent. He found a significant earimt of variance in the ratings 
attributable to differences in collegiate degrees held. His procedures 
night be questioned, however, as he apparently sunsad the coaposite scores 
for each section of the rating sheet in order to cd>tain the figures which 
were used in the analysis. To Justify such a practice, it is necessary to 
make tenable the assuaption that all iteas on the form can be assigned 
equal weight. 
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Carter^ used the State Evaluation fbrm to distinguish between 
high and low merit-rated junior college teachers. He then studied the 
relationships between these ratings and personal, educational, and 
experience factors. He found few significant relationships. He, like 
Wursbach, also used composite scores as his measure of teacher cosipetence. 

Gerlock^ compared the ratings given professionally and pro- 
visionally certificated first year teachers. He did separate item-by-item 
analyses and found that those teachers meeting professional certification 
standards scored significantly higher on some items, particularly those 
relating to teaching skill. 

Four additional studies dealing with factors which might warrant 
consideration in interpreting data collected with the Florida evaluation 
form have also been conqtleted. McTeer^ investigated the hypothesis that 
teacher-principal likenesses are a factor in teacher rating. He assuaied 
that the more similarities which existed between the teacher and the 
principal, the higher would be the teacher's rating. Factors which he 
studied include age, grade point average, highest degree held, and amount 
of professional work as an undergraduate. He found soom significant 
correlations, but in no case did the likeness factors account for a sub- 
stantial amount of the variance. 8obm shortcomings of his study are that 
he apparently cond>ined scores on scale items and did not account for dif- 
ference in the general level of scores assigned by different principals. 

Packer^ was interested in the sensitivity of the instrusient to 
**self-others acceptance" which he deemed an essential trait for an effective 
teacher. He found very little relationship between this construct and the 
principals' rUtings and concluded that either the principals were not suf- 
ficiently familiar with the teaching situations or they were not appreciative 
of "self-others acceptance" as a determinant of effective teaching. 



Studies Using the 
Current Version 



Two recent studies by Daniel employed the current version of 
the Florida lhacher Evaluation Form. The first employed the analysis of 
variance technique to determine whether certain factors could account 
statistically for differences in ratings assigned to a teacher on any 
items on the form.' The factors tested were (a) subject or grade being 
taught (i.e. , Is there any difference between ratings assigned to teachers 
of one grade and those assigned to teachers of another grade?), (b) evaluator 
(i.e.. Is there any difference between the level of ratings assigned by one 
evaluator and those assigned by others?), (c) certificate rank (i.e., edu- 
cation level), (d) number of different preparations which the teacher must 
make, (e) sex of the principal, and (f) age of the principal. A summary 
of significant effects is given on Table 3. It should be recognized that 
this table suranarizes the results of several separate analyses. From the 
table, it is apparent that, among those effects tested, the only one which 
contributed significantly to the variance in ratings was the evaluator 
effect. This can be interpreted two ways; either some principals tend to 
rate their teachers higher than do other principals, or some principals 




40 



TABI£ 3*— CcnparlsoKus of ratings assigned to the same teachers using 
the official Florida Teacher Evaluatlcn Fom 
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have better teachers in their schools than do others* A subse^nient of 
studies has been begun to investigate this problem further* The value 
of. such efforts is liMited, however, because technical deficiences of 
the evaluation form place limitations on interpretations which can be 
made from statistical manipulations of the data therefrom. This problem 
is discuised later in greater detail. 

A second study employing the present form desls with the 
objectivity of ratings assigned by principals.® Objectivity is defined 
as the ertent to which independent ratings of one teacher completed by 
two different evalndtors are in agreement. In this study, ratings by 
assistant principals and by supervisors were conquered with those sub- 
mitted by principals, these ratings are summarised in Table 4. It was 
found in both the sampli nf sisistant principals snd the sample of 
luperyiiorai the principal and the other rater agree about two-thirds 
of Cmih the diittibut ions of retlnge observed, agteement could 

usually be expected ehout half of the time by chance.’) 



Evaluating the Florida Teacher 
Bvaiuatidn ?wgram 

Evaluating the Florida teacher evaluation program is difficult, 
since there is no specific purpose which the program has been designated 
to fulfill. It might be assumed, however, since the evaluation forms 
are devised for machine processing end since routines have been established 
to tabulate and summarise the data, thst there hss been some intention to 
secure comprehensive information relating to the quality of teaching in 
the 9Mte* . thie is informstion which might be used in developing weys to 
improve the educational program. Further evidence of such sn 4ntentlon 
can be found,.in the proposal for a "multi-state project to improve teacher 
education*' submitted to the U. 8. Office of Education in 1985. It was 
proposed thst the Florida State Department of Education use itt teacher 
evaluation datn along with other data to determine needed services i|i the 
areas of pre-service and in-service teacher education* 

When viewed in this framework, the Florida teacher evaluation 
program displays striking inadequacies. The technical deficiencies in the 
instrud^t are one example* The extreise skewing in the distribution of 
ratings assigned on the form prohibits the utilisation of parametric^ 
statistical techniques when analyzing the data. Thus , inferences drawn 
from the data using standard scientific procedures must be interpreted 
With caution. The lack of homos cedast id ty which can result from the skewing 
often mal^s It inadvisable to calculate Pearson product -moment coefficients 
of correlation to provide indicies of the relationship between ratings 
received and other variables. 

The principal handicap which is imposed by a rating scale with a 
limited number of steps results from restrictions upon amount of information 
which the scale can provide. The optimum number of scale points is that 
which makes the maximum use of the observer's discriminative powers. 
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This point is reached when the ptio between true variance and error 
variance is maximum. Guilfprd^^ .reviews a number of studies which deal 
with the optimum number of > steps to be used in a rating scale. He con- 
cludes that the numbep is' usually greater than seven and may, in certain 
situations, be as high as twenty-five. The Florida form employs scales 
with three steps, and in Some cases ti^o. 

The Florida evaluation form also appears to have some con- 
ceptual deficiencies. Particularly notable is the great emphasis placed 
upon general attitudes or personality traits which are deemed to be con- 
ducive to effective teaching and the small amount of emphasis upon 
behaviors which are actually involved in teaching. The only items on 
the form which definitely fall into the latter category are "Uses 
instructional materials and lesson plans effectively," "Develops pupil 
interest and eagerness to learn," and "Maintains pupil control." These 
are the only items on the form which can logically be employed ss measures 
for evaluating teaching. The other items are characteristics of teachers 
which are more appropriately viewed as possible predictors of teaching 
behavior. 



To summarize, it appears that, because of deficiencies of the 
evaluation instrument, the Florida teacher evaluation program can provide 
very little information which can be used to improve education in Florida. 



HE ACTIONS TO STUDIES USING DATA COLLECTED 
WITH THE CURfiENT FORM 

James C. Impara, Research Consultant 



Mr. Daniel has asked me to react to the information provided 
in the handout which has been distributed. 

Recall the pilot studies by Mr. Daniel in which several 
(96 to be exact) analyses of variance were reported. It is noted that 
the value of these attempts is limited due to technical deficiencies 
in the teacher evaluation form. There are two points which I think 
are important, statistically, with respect to these studies. 

1. As was pointed out to the presentation, there are 208 
cells which were tested for significance including interactions. 

If only one test was done per cell and if we assume that the data 
were appropriate for this type of design, then at least 10 cells 
could be expected to show significance at the .05 level due to chance 
alone and at least 2 cells at the .01 level. Since it appears that 
more than one test was possible for each cell this greatly increases 
the probability of obtaining significant differences purely by chance. 

2. The next point is with respect to the technical deficiencies 

of the instrument when used in the analysis of variance design. With 
only 3 choices of ratings: Superior, Satisfactory, or Poor, the dis- 

tribution obtained is likely to be sever ly skewed, as is the case with 
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Mr. Daniel's data. The akewneas, by itself, is not a severe probltSi. 

The problems are: (1) that little or no variance is even present due 

to the limited number of ratings; and (2) the scale of ratings is, at 
best, ordinal which also presents difficulties in the interpretation 
of results. These same reasons would apply to most parametric and 
many non parametric tests of significance, thus limiting the use of 
this form in research on teacher evaluation. 

Mr. Daniel's second study was about the "objectivity of 
ratings". At first glance it would appear that a series of correlations 
would have been more appropriate than the percent of agreement that 
is shown in Table 4. On closer inspection of the data, however, these 
percents are perhaps more appropriate than correlations. 

There are basically (but not exclusively) 7 methods of 
obtaining a coefficient of correlation- (Pearson - Rank Order - Tatra - 
Contingency - Biserial - Point Bis--ETA). Only two of these might 
have been appropriate ter Mr. Daniel's purposes. These are the Pearson 
P.M. and the ETA Coef. The basic assumptions of the Pearson is linearity 
of relationship, and homoscedasticity of scatter. Mr. Danielle data met 
neither of these basic assumptions. Due to the evaluation form the 
continuity of ratings is questionable. The ETA coefficient for nonlinear 
data is, at best, at tenuous device for determining degrees of relationship. 
Hence none of the better known correlation techniques were appropriate 
in determining a measure of objectivity. 

Due to these shortcomings in the evaluation instrument it was 
determined that Mr. Daniel's purposes could be adequately met by using 
percent of agreement as a measure of objectivity. 
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THE MEASUREMENT AND EVALUATION OF TEACHING: 
A CONCEPTUALIZATION Ot A FLAN FOR USE IN 
STATE EDUCATIONAL LEADERSHIP* 

by K. Fred Daniel 



Au extensive study dealing with the problem of evaluating 
teaching was recently carried out in the Florida State Department of 
Education under the Ihilti-State Teacher Education Project. This study 
was instituted to delineate the cond^^ions under which statewide pro- 
grams for measuring and evaluating teaching can yield the most useful 
information for inqplementing state educational leadership. The leader- 
ship under consideration reir^ted particularly to the following state 
services: education of teachers, certification of teachers, and accredi- 

tation of schools. 

I^fo methods were eiq>loyed to identify possible methods for 
measuring and evaluating teaching: a synthesis of relevant literature 

and a questionnaire study. The following section and the later discussion 
of approaches for evaluating teaching are based upon the synthesis of 
literature. 



The Significance and Ambiguity of 
Information on Teaching 

. 

. Several different types of activities are carried out by state 
governments in their efforts to provide for high quality teaching in 
elementary and secondary schools. These include (1) developing and 
administering procedures to insure the quality of programs for the pre- 
paration of teachers, (2) developing and administering standards which 
will allow only qualified people to hold teaching positions, and (3) 
developing and administering programs for promoting teacher growth and 
eliminating substandard teaching situations. 

The significance of information on teaching for use in state edu- 
cational leadership <Ierives from the fact that it constitutes the most 
relevant t]rpe of information for evaluating these activities. State edu- 
cational leadership consists of devising ways to iq>rove these activities. 
A system for collecting information on the teaching which is taking place 



-k 

This report is based upon a more comprehensive document by the 
same title. The complete report is available from the Division of Teacher 
Education, Certification and Accreditation, Florida State Department of 
Education. 
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in the schools could provide the feedback which is necessary to ioipleaient 
leadership to bring about the improvement of the state programs. 

It is noted that virtually no systematic use of such infonsation 
is presently made by state governments. This could be attributable to 
the fact that much information on teaching which has been or mi^t be 
collected is ambiguous. The sources of such ambiguity are nuswrous. 

There is no widespread agreement as to what criteria should be used 
for evaluating teaching. Therefore, confusion may result when two 
people analyze the same evidence because they are using different criteria. 
Much research has been conducted to determine the factors which can account 
for effectiveness or ineffectiveness in teaching. This has generally not 
been fruitful. A further complicating factor relates to the tenuous 
relationship between teaching and the subsequent behavior of pupils. The 
influence of the teacher is only one of the many effects which contribute 
to pupil learning. 



State Programs for Evaluating Teaching 
in Pennsylvania . Hawaii and Florida 

A questionnaire survey conducted indicates that at present there 
are only three state programs prescribed for evaluating teaching in the 
United States. These are found in Pennsylvania, Hawaii, and Florida. 

The State of Pennsylvania has prescribed a form and procedures 
for rating teachers. Satisfactory ratings on the form are required in 
order for a teacher to be eligible for tenure and in order for him to 
have this certificate renewed. An unsatisfactory rating, along with 
supporting evidence, provides a legal basis for dismissing a tenure teacher 
on grounds of incompetence. The Pennsylvania program is not designed to 
provide information which can be used in making decisions relative to 
teacher education, certification and accreditation. 

The statewide evaluation program in Hawaii has as its states 
purpose the improvement of teaching in the schools of that State. It is 
a program similar to one which might be developed in a large school district. 
The stete prescribes a form and procedures; principals observe the teachers 
and, following the observations, review the ratings in a conference with 
the teachers. The program for evaluating the work of tenure teachers per- 
forms no official regulatory function. However, as in Pennsylvania, satis- 
factory ratings are required for a probationary teacher to become eligible 
for tenure. The Hawaii program is not designed to provide information which 
can be used to evaluate the effectiveness of state educational policies and 
programs . 



In Florida, all teachers are evaluated annually using a form pres- 
cribed by the State. The results are place on file in the State Departsient 
of Education in a manner which allows them to be summarized and analyzed 
using electronic data processing equipment. While no official purposes 
have been specified for this program, it is organized in a manner which 
should provide the kind of information which could be used to make decisions 
regarding state policies for improving teaching. However, an examination 
of data collected indicates various shortcomings which considerably restrict 
interpretations that can be made using this information. 
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Approache'8 to Evaluating Teaching 



The proceaa of evaluation is conceived as an operation whereby 
evidence is compared with criteria. Consequently, the nature of the 
evidence which is relevant is determined by the criteria. Three types 
of criteria— product , process, and presage— have been identified. Product 
criteria comprise the outcomes toward which teaching ia directed. Process 
criteria are those types of teacher behavior believed to be inherently 
worthwhile, at least in given situations. Presage criteria are made up 
of those traits or experiences which are thought to be fundamental to 
certain facets of teaching performance (which can be evaluated employing 
process criteria) or to achieving certain outcomes (which can be evaluated 
employing product criteria). Either the product type or the process type 
can be used in evaluating; presage criteria are, in reality, predictors. 

The selection of criteria constitutes a value judgement which 
may be quite arbitrary. However, a person who is familiar with the situation 
to which the criteria will apply is normally in a better position to make 
a judgement as to what constitute acceptable criteria. 

The process of collecting evidence and comparing it with criteria 
can be viewed along three dimensions. They relate to (1) the nature of the 
situation in which the evidence is collected (varying from "normal" to 
"constructed"), (2) the agent responsible for collecting the evidence (with 
the performer hiosself represented at one extreme and observer at the other), 
and (3) the point at which the evidence is compared with the criteria 
(varying from "%diile observing" to "independently"). 

There are several techniques for collecting evidence and comparing 
it with criteria which can be used. These include self reports and self 
evaluation, rating and rating scales, systematic observation with evalua- 
tion performed independently, and testing. There are conditions under 
which each of these techniques might be applicable in a statewide program 
for obtaining information to implement educational leadership. 



A State Program for Collecting and Analyzing 
Information on Teaching 



The effectiveness of a program for collecting information on 
teaching to implement state educational leadership depends first upon three 
conditions which must be met by the information used: relevance, interpret- 

ability and reliability. Relevance refers to the relationship between the 
information collected and the circumstances which state educational leadership 
is intended to effect. Because this leadership is aimed primarily at helping 
local school districts to carry out programs which meet the needs of their 
citizens, it is proposed that the most relevant information can be obtained 
from carefully designed local evaluation programs. This, however, could 
lead to problems of interpretability, the second essential condition for 
an effective program. 

Information which is interpretable can be sumnarized and analyzed 
in ways which will yield information that can be used for the desired pur- 
poses: in this case, for the implementation of state education leadership. 

Attempting to combine data collected in different places under different 
circumstances is certain to bring about problems of interpretability. To 
deal with these, it is proposed that communication be developed between 
^ schools using similar criteria, that pilot programs for the development 

ERIC 
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Qf exemplary evaluation ayatems be establiahed and the reaulta widely 
diaaeminated , and that a taxonomy of criteria including the core or 
aioat baaic elementi which are included in evaluation programa through- 
out the atate be compiled. The taxonomy could be the atarting point 
for the developamnt and atandardiaation of inatrunenta for meaauring 
varioua element a of teaching. 

Reliability, the third eaaential condition for an effective 
program ref era to the conaiatency between the information on teaching 
which ia collected and traita, behaviora, or conditiona which that 
tofomation ia aaauiaed to repreaent. Reliability reaulta when carefully 
developed inatnanenta and procedurea are uaed. 

The information would be collected in a central location and 
atored in a form which would allow retrieval and analyaia uaing electronic 
data proceaaing e<juipment. Six different filea are propoaed for col- 
lecting and atoring the information. They would contain (1) information 
on teacher education program criteria, (2) information on achool program 
criteria, (3) teacher atatua information, (4) achool atatua information, 

(5) teaching performance information, and (6) pupil performance information. 

The general procedure for analyzing the data involvea two phaaea. 
The firat conaiata of aelecting from the filea the data for analyaia. 

The data could coaie from any one file or any combination of filea. The 
aelected data would be incorporated into a new file. The aecond phaae 
conaiata of the analyaia, itaelf. Thia would utilize deacriptive or 
inferential atatiatical techniquea. 



Reflectiona 



The atudy propoaea the development of an information ayatem which 
ia capable of providing facta which can be used by the state in strengthening 
educational leadership. The system is designed to allow the state to make 
decisions relative to policies for teacher education, teacher certification, 
and school accreditation on a aiore -rational basis. 



Broadening the State Conmitment 
to Institutional Research 

The main proposal of the study could also be couched in different 
temis. A proposal for the development of a system to provide data which 
makes it possible for administrators to make better decisions is a pro- 
posal for institutional research. Institutional research represents a 
rational, rather than arbitrary, approach to decision-making. It is widely 
practiced in certain phases of state school administration. On matters 
relating to educational finance, such as cost of certain programa or tax 
paying ability in certain areas or of certain segments of the population, 
institutional research is the rule rather than the exception. Institutional 
research is also practiced extensively in anticipating needs for new 
facilities. However, very little use is made of institutional research in 
developing policies for teacher education, certification, and accreditation. 
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The primary proposal in this study is that attempts be made 
to take a rational approach to these matters, as is typically taken 
in the domains of finance and facilities. Optimum decision# are more 
to result when as imich relevant information as possible can be 
brought to bear. 



Codifying the Wisdom of 
Educational Practitioners 

In order to obtain relevant information the study proposes a 
concerted effort to codify the wisdom of educational practitioners. 

The results of such a program would have significance far beyond the 
inmediate concerns of providing the state government with information 
which it can use. 

The proposal for codifying the knowledge of practioners 
resulted from the concept that evaluation must be based upon criteria 
the stipulation that neither the teaching processes to be employed nor 
the teaching products to be sought should be dictated by the state to 
local school personnel. This means that, if evaluation is to take place, 
local school personnel must establish the criteria. The establishment of 
criteria by such personnel would constitute a codification of their pro- 
fessional wisdom. 

The resulting statements would set forth the factors which. In 
the judgement of practicing educators, determine sound educational practice. 
It would not be a superficial document as might result from a simple survey 
or Q-sort. Since the statement of criteria would form the official basis 
for evaluating the work of teachers, it would be reasoned out in a manner 
which would strike for the essence of the issues. As the criteria were 
applied, the codification would be continually re-evaluated, revised, and 
• As communication and collaboration between schools increased, 
the body of knowledge developed by professional practitioners would become 
more refined. 

Lortie declares that a codified body of knowledge representing 
the best thinking of skilled professional practioners is something which 
is lacking in the field of education. This is not the case, however, in 
the profession of medicine, law and architecture. He notes that, while 
the professional subject matter for the teacher contains contributions 
from philosophers and psychologists, it does not include a body of know- 
ledge codified by educational practitioners. In medicine, law, and 
architecture there are such courses, based upon the ciimulated wisdom of 
practitioners . ^ 

l*oi^tie also observed that there exists a chasm between schools 
of education and teachers in the elementary and secondary schools. He 
asserts that ceachers do not attribute the same importance to professional 
schooling as to physicians, lawyers, and architects.^ Research results 
documenting the gap between education professors and teachers are reported 
by Joyce ^ who found that attitudes toward teaching held by beginning 
teachers are more in agreement with those held by the general public than 
with those held by professors of education. 
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The codifying by practitioners of a body of practical know- 
ledge could contribute to the ioiproveiiient of teacher education and also 
to the narrowlhg of the breach between practitioners and professors. This 
knowledge would give beginning teachers invaluable support as they com- 
vence their professional duties. Excerpts fron this body of knowledge 
would certainly be adopted by schools or departments of education and 
integrated into their programs. Hopefully, the ultisiate result would be 
the wedding of the best elements from both the professors and the 
practitioners. One effect of this would be a more prominent place for 
elementary and secondary classrooms in programs of teacher education. 

It is often stated that professional education for teachers 
should becosm SK>re closely associated with the classroom. Typical recom- 
mendations are that longer student teaching periods be established, that 
internship programs be developed, that students be assigned case studies 
or other projects which will bring them into the schools, or that instructors 
in graduate courses slant their presentations more directly toward the 
problems of the teachers enrolled. 

It is seldom, if ever, suggested that the wisest and most skill- 
fol*teadhers perfoim the difficult task of codifying their professional 
wisdom so that it can benefit both them and their associates. Tet, it 
seems unnecessary that each first-year teacher should have to begin anew 
to develop such a body of professional wisdom. It also seems unreasonable • 
that a. professor of education should have to relate his psychology or 
to dhi^e^inaaiples when a of professional informat ton 
by mfucational prictit toners sd^t be available to form a basis 
'1^ die^ssiOT. The codification of the knowledge of practicing educators 
<^id serve to bring teacher preparation and teaching closer together. 

It P^^ ^br an easier induction of the beginning teacher and 

a. b Vri a f coananiication between teachers and education professors from 
wlitoh they could proceed to work together to improve teaching. 



Some Side Effects 

" “I ' * - 

There are at least two areas not yet mentioned in which imple- 
mentation of the above general proposals would contribute to the improve- 
ment of teaching. These areas represent functions which are, in fact, 
more fundamental in improving teaching than are the state services toward 
which the infonsation system is directed. The first relates to research 
on teaching; the second to stimulation of local leadership for the improve- 
ment of instruction. 

In the area of research, the large volume of carefully collected 
measures and evaluations of teaching would make data available for numerous 
correlation studies or other investigations employing pre-experimental or 
guasi-experimental designs. These would include studies probing relation- 
ships between various status characteristics of teachers and various types 
of performance of teachers or between various types of performance of 
teachers and various teaching products. 
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It is also expected that Impleiiientatlon of the system would 
lead to impifoved understanding and respect by school personnel of the 
researcher's approach. This would result from continual efforts by teachers 
and administrators to define their criteria and to eiq>loy measures for 
determining the extent to which they have been met. This is the same 
basic problem ^ich the research worker encounters repeatedly throughout 
his career. 

The understanding which the school people are expected to 
develop should make the schools more accessible for use as laboratories for 
experimental studies. Moreover, it seems likely that their greater 
sophistication would contribute to better experimental controls and better 
research results. 

Instructional leadership at the local level would also be affected 
by the greater understanding of research. The search for criteria and 
for methods of applying them could make school personnel more alert and 
receptive to research results which mi^t provide assistance or guidance. 

However, the greatest stimulus to instructional improvement 
within the local schools would almost certainly result from the initiative 
taken by local school personnel to define and redefine the criteria which 
they wish to enqtloy in evaluating teaching. With definite criteria at 
hand and definite evidence as to the extent to which the criteria are being 
met, more decisive measures can be taken to improve teaching. 



Conclusion 

The study described in this paper was instituted to delineate 
the conditions under which statewide programs for measuring and evaluating 
teaching Cr<n yield the most useful information for implementing state edu* 
cational leadership. After reviewing relevant literature and current 
practices, the study proposes a comprehensive information system designed 
for yielding information to allow decisions relative to teacher education, 
teacher certification, and school accreditation to be made on a more 
rational basis. 

Of particular note, however is the fact that the implementation 
of the system would provide some additional benefits which are, in reality, 
more significant for improving teaching than state regulations and policies. 
These include (1) increasing the potential contributions of practitioners 
to teacher education, (2) stimulating educational research and the appli- 
cation of research finds, and (3) stimulating local leadership for the 
improvement of instruction. 
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PROGRAM 



Opening Remarks 



9:00 ••m. Welcome 



"The M-STEP Idea” 



Honorable Floyd T. Christian 
Howard E. Bosley 



Ideas Being Implemented by M-STEP in Maryland . Michigan . 
South Carolina . Utah . Washington , and West Virginia 



9:30 a.m. "Videotapes for 

Teacher Education” 

"Centers for Student 
Teaching” 



George W. Hopkins 
Vere A. McHenry 

Jerry B. Chapman 
John B. Himelrick 
Arthur P* Kulick 



"Coordinating Preservice 
and Inservice Teacher 
Education Programs” 



William H. Drummond 



11:00 a.m. INTERMISSION 



11:15 a.m. 



"Can the M-STEP Ideas be 
Adapted to Florida's 
Needs?” 



(Small Group Discussions) 



11:50 a.m. 



Questions for M-STEP Panel 
Members 



12:30 p.m. LUNCH 
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The Florida M-STEP 




2:00 p.m. "Using Information to Improve 
Educational Leadership" 

"Information on Teachers 
and Schools Collected by 
the State Department of 
Education" 

"The State Department of 
Education Data System" 

"Using the Data We Have" 



"An Analysis of the Florida 
Teacher Evaluation Data" 

"A 'Model' Teacher Evaluation 
System" 



4:00 p.m. ADJOURNMENT 




W. Cecil Golden 



Simon A. Kinsey 
Ray V. Pot tor f 
Laymon Gray 



L. Everett Yarbrough 

K. Fred Daniel 
Wil C. Robinson 

K. Fred Daniel 
James C. Impara 



K. Fred Daniel 
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V 






CONFERENCE PARTICIPANTS* 



Ted Andrews, Bureau of Teacher Education, New York State Department 
of Education (5) 

William H. Banka, Jr. j Jackaonville University (2) 

James A. Bax, Florida Atlantic University, Boca Raton 
Robert D. Binger, Conaul^ant, Science, Florida State Department of 
Education (5) 

Garth K. Blake, Florida Btate University 
Howard E. Boaley, Director, M>STEP, Baltimore, Maryland 
Jerry E. Chapman, M-STEP Coordinator, Michigan State Department of 
Education (4) 

CharlOB E. Chick, Executive Director, Divialon of School Adminlatration, 
Florida State Department of Education (2) 

Thomas Culton, Consultant, Curriculum, Florida State Department of 
Education (4) 

K. Fred Daniel, M-STEP Coordinator, Florida State Department of Edu- 
cation 

William H. Druanond, M-S1EP Coordinator, Washington State Department 
of Education (1) 

Deane Ducar, Coordinator, Kemp Mill Education Center, Silver Springs, 

Maryland H) 

Roaa L. Duhart, Booker T. Washington High School, Tampa (representing 
Florida TEPS Commission) (5) 

Roy E. Dwyer, University'of West Florida, Pensacola (representing the 
Florida Unit, Association for Student Teaching) (1) 

Ken M. Eaddy, Coordinator, Vocational Research, Florida State Depart- 
ment of Education (1) 

M. Mitchell Ferguson, Executive Director, Division of Teacher Education, 
Certification and Accreditation, Florida State Department of Education (2) 
Gordon Foster, University of Miami, Coral Gables (3) 

Charles K. Franzen, Associate Director, M-STEP, Baltimore, Maryland (5) 
•Marshall Frinks, Jr., Coordinator, Regional Curriculum Project, Florida 
State Department of Education (3) 

J. L. Gant, General Consultant, Curriculum and Instruction, State Depart- 
ment of Education (5) 

William E. George, Consultant, School Accreditation, Florida State Depart- 
ment of Education (3) 

Wm. Cecil Golden, Assistant Superintendent, Teacher Education, Certification 
and Accreditation, Florida State Department of Education 
John B. Himelrick, Sr. , M-STEP Coordinator, West Virginia State Depart- 
ment of Education 

George Hopkins, Director, Division of Teacher Education and Certification, 

South Carolina State Department of Education (1) 

James W. Hosch, III, Research Associate, Florida State Department of 
Education (4) 

James C. Impara, Research Consultant, Florida State Department of Education (4) 
Boyd Israel, M-STEP Coordinator, South Carolina State Department of Edu- 
cation (2) 




♦The number In parenthesis after each entry indicates the number of the 
discussion group in which that individual participated. 
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Leon King, Secondary Curriculum Coordinator, Hillsborough County Schools 
(representing the Teacher Education Advisory Council) (2) 

Arthur P. Kulick, M**STEP Coordinator, Maryland State Department of Edu- 
cation (4) 

William L. Maloy, Florida State University (2) 

Robert D. Martin, Florida State University 

Vere A. McHenry, M-STEP Coordinator, Utah State Department of Education 
Harland Merriam. Stetson University, DeLand (1) 

Sam H. Moorer, Board of Regents Office for Continuing Education (4) 

John A. Outterson, University of Tampa (5) 

Robert A. Pace, Jacksonville University (2) 

Ray V. Pottorf, Director, Teacher Certification, Florida State Depart- 
ment of Education (4) 

David Ramsey, Consultant, Science, Florida State Department of Education (3) 
Hubert Richards, Consultant, General Instruction, Florida State Department 
of Education (2) 

Hil C. Robinson, Programming Specialist, Florida State Department of Edu- 
cation 

Burke B. Scisson, Consultant, General Instruction, Florida State Depart- 
ment of Education (2) 

John Shadgett, Board of Regents Office for Continuing Education (3) 

Bert Sharp, University of Florida, Gainesville (4) 

Evelyn Sharp, Bethune-Cookman College, Daytona Beach (3) 

Rodney P. Smith, Jr., Consultant, Language Arts, Florida State Department 
of Education (3) 

Sister Ann Thomas, Professor of Education, Barry College, Miami 
Ray Tipton, Florida Education Association 

John H. Waters, Coordinator, Title III, NDEA, Florida State Department of 
Education (1) 

W. W. Wharton, Director, Scholarships and Loans, Florida State Department 
of Education (5) 

William H. Woodham, Jr., University of West Florida, Pensacola (3) 



THE MULTI-STATE TEACHER EDUCATION PROJECT 



Project Coordinating Board 

H. T. Boaton, Aaalatant Superintendent, Certification and Accredita- 
tion, Maryland State Departnent of Education 
Eugene Rlchardaon, Consultant, Higher Education and Certification, 
'Michigan State Departnent of Education 

®*®*^E* Hopkins, Director, Division of Teacher Education an d Certifi- 
cation, South Carolina State Departsient of Education 
N. Blaine Vlsters, Administrator, Division of Teacher Personnel, Utah 
State Board of Education 

Wendell C. Allen, Assistant Superintendent for Teacher Education and 
Certification, Washington State Departomnt of Education 
Genevieve Starcher, Director, Division of Teacher- Preparation and Pro- 
Standards. Wesc Virginia State Departnent of Education 
W. Cecil Golden, Assistant Superintendent for Teacher Education, Certifi- 
cation and Accreditation, Florida State Departnent of Education 



State Project Coordinators 



Jerry E. Chapman, Michigan 
William H. Drummond, Washington 
John B. Hlmelrlck, W6st Virginia 

R. Fred Daniel, 



Boyd Israel, South Carolina 
Herman E. Behllng, Jr., Maryland 
Vere A. McHenry, Utah 
Florida 



Central Office Staff 

Howard E. Bosley, Director Charles K. Franzen, Associate Director 

The Project Central Office Is located In Suite 205, 1101 St. Paul Street 
Baltimore, Maryland 21202 * 



